Moisture sensor and windshield fog detector

ABSTRACT

A control system is disclosed for automatically detecting moisture on the windshield of a vehicle. The automatic moisture detecting system includes an optical system for imaging a portion of the windshield onto an image array sensor, such as a CMOS active pixel sensor. The voltage of each of the pixels which represents the illumination level is converted to a corresponding gray scale value by an analog digital converter. The spatial frequency composition of the gray scale values are analyzed to determine the amount of rain present in order to provide a control signal to control the operation of the windshield wipers of the vehicle as a function of the amount of moisture present. The system is also adapted to detect the level of fog both on the interior of the windshield as well as the exterior of the windshield.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.10/999,175 filed Nov. 29, 2004, now U.S. Pat. No. 6,946,639, which is acontinuation of U.S. patent application Ser. No. 10/289,239 filed Nov.6, 2002, now abandoned, which is a continuation of U.S. patentapplication Ser. No. 09/878,799 filed Jun. 12, 2001, now U.S. Pat. No.6,495,815, which is a continuation of U.S. patent application Ser. No.09/592,896 filed Jun. 13, 2000, now U.S. Pat. No. 6,262,410, which is acontinuation of U.S. patent application Ser. No. 09/347,093, filed onJul. 2, 1999, now U.S. Pat. No. 6,097,024, which is a continuation ofU.S. patent application Ser. No. 08/931,118, filed on Sep. 16, 1997, nowU.S. Pat. No. 5,923,027, the entire disclosures of which areincorporated herein by reference.

TECHNICAL FIELD

The present invention relates to a system for automatically detectingthe presence of moisture on a surface such as the surface of awindshield of a vehicle, in order to automatically actuate the vehicle'swindshield wipers and/or defroster or defogging system.

BACKGROUND ART

In conventional windshield wiper systems, the windshield wipers areactuated based upon the elapsed time between wipes rather than themoisture level on the exterior of the windshield. During conditions ofrelatively consistent rainfall, for example, the time interval can beadjusted to correspond to the amount of time in which the rainaccumulates to the point of the desired visibility level. Unfortunately,the rate of rainfall may vary dramatically over a given period of time.Additionally, traffic conditions may also cause varying amounts of rainto fall on the windshield due to traffic conditions, such as a passingtruck or the like. As a result, during such conditions, the wiper timeinterval must frequently be adjusted, which can be cumbersome.

Various systems are known which automatically control the intervalbetween wipes of the windshield wipers based upon moisture on thevehicle windshield. In some known systems, various coatings are appliedto the vehicle windshield. Electrical measurement of those coatings isused to provide an indication of the moisture content on the windshield.Unfortunately, such methods require relatively expensive processes whichmake such system commercially non-viable. Other systems forautomatically sensing the moisture content on a vehicle windshield arealso known. For example, optical systems are known which measure thedifference of reflected light of a dry windshield versus a wetwindshield. Unfortunately, the optical method is susceptible tointerference from external light sources and thus provides inadequateperformance. Other known systems must be adhered to the windshield whichcomplicates the windshield replacement. As a result of suchcomplications, moisture sensors are rarely found on vehicles.

Another system for automatically detecting the moisture content on awindshield is disclosed in Japanese Laid Open Patent Application No.Hei>(1995)-286130, which describes the use of a charge coupled device(CCD) image sensor to image a portion of the vehicle windshield in orderto detect raindrops. The system described therein computes the sum ofthe differences between each pixel and the average of all pixels.Unfortunately, headlamps of oncoming vehicles will create a bright spotin the image which would be difficult to completely blur and likely beinterpreted as rain. Moreover, in order for such a system to workeffectively, images from the distant scene must be completely blurred.Otherwise, there will be dark and light regions in the distant scene.Although there is no optical system disclosed in the Japanese laid openpatent application for accomplishing this objective, it would be verydifficult to develop an optical system to completely blur an oncomingheadlamp. Failure to blur oncoming headlamps could cause falsetriggering of the system disclosed in the above-identified Japanese laidopen patent application.

Another problem with automatic rain detection systems is the inabilityof the system to detect the operation of the windshield wipers. Incertain cold climate conditions, the windshield wipers are known tofreeze to the windshield. In such a situation, since the moisture is notbeing removed by the wipers, an automatic rain sensing device wouldcontinuously command the wipers to actuate, even though the wipers arefrozen to the windshield, potentially damaging the windshield wipersystem.

Another known problem with known systems is the inability to detect fogon the interior and exterior of the windshields. As mentioned above,automatic moisture detection systems, such as disclosed in theabove-identified Japanese laid open patent application, are based uponthe ability to detect raindrops on the windshield. When a uniform fog ormist covers the vehicle windshield, systems, such as the systemdisclosed in the Japanese laid open patent application, are unable tosense such moisture on the exterior of the windshield. As a result,during such a condition, the windshield wipers will have to be manuallyactuated, thereby partially defeating the purpose of an automatic rainsensor and windshield wiper control system making the feature a lot lessdesirable.

In other situations, fog develops on the inside of the windshieldindependent of the moisture content on the exterior of the windshield.In such a condition, automatic rain sensing systems, such as disclosedin the Japanese laid open patent application, are unable to detect themoisture content on the exterior of the vehicle windshield until afterthe fog on the interior of the windshield is cleared. In such acondition, a defroster or defogger system would have to be manuallyactuated to remove the interior fog on the windshield. The automaticrain sensor would not be operable during such a condition until the fogon the interior of the windshield is sufficiently cleared.

The desirability of having automatic rain sensing is to have a systemwhich automatically controls the windshield wipers during typicalclimatic conditions, such as rain, snow and fog. When the wiper systemhas to be operated manually during such typical conditions, such afeature becomes undesirable.

DISCLOSURE OF INVENTION

It is an object of the present invention to provide a system whichsolves the problems of the prior art.

It is yet another object of the present invention to provide a systemfor automatically detecting the moisture content on the windshield of avehicle.

It is yet another object of the present invention to provide a systemfor automatically detecting moisture on the windshield of a vehicleduring common climatic conditions, such as rain, snow and fog.

It is yet a further object of the present invention to provide a systemfor automatically detecting the presence of fog on the exterior of awindshield.

It is yet another object of the present invention to provide a systemfor automatically detecting the presence of fog on the interior of awindshield.

It is yet a further object of the present invention to provide a systemfor automatically sensing the windshield wipers crossing a portion ofthe windshield.

Briefly, the present invention relates to a system for automaticallydetecting moisture on the windshield of a vehicle. The automaticmoisture detecting system includes an optical system for imaging aportion of the windshield onto an image array sensor, such as a CMOSactive pixel sensor. The voltage of each of the pixels, which representsthe illumination level, is converted to a corresponding gray scale valueby an analog to digital converter. The gray scale values correspondingto the image are stored in memory. The spatial frequency composition ofthe gray scale values is analyzed to determine the amount of rainpresent in order to provide a control signal to control the operation ofthe windshield wipers of the vehicle as a function of the amount ofmoisture present. The system is also adapted to detect fog both on theinterior of the windshield as well as the exterior of the windshield. Byproviding a system for automatically detecting the presence of fog onthe interior and exterior of the windshield, serious performancelimitations of known automatic rain sensors during typical climaticconditions are eliminated.

BRIEF DESCRIPTION OF DRAWINGS

In the drawings:

FIG. 1 is a physical diagram which illustrates a vehicle windshield andan attached rearview mirror illustrating a system in accordance with thepresent invention;

FIG. 2 is an enlarged view of a portion of the system for sensingmoisture on the exterior of a windshield in accordance with the presentinvention;

FIG. 3 is a physical diagram of a system in accordance with an alternateembodiment of the invention for sensing fog which illustrates theprojection of a beam of light onto the windshield for fog detection;

FIGS. 4 a and 4 b are computer-simulated spot diagrams which illustratethe performance of the optical system in accordance with the presentinvention during moisture and non-moisture conditions, respectively;

FIG. 5 is a flow diagram for the system in accordance with the presentinvention; and

FIG. 6 is a block diagram of the system in accordance with the presentinvention.

BEST MODE FOR CARRYING OUT THE INVENTION

An automatic moisture sensing system in accordance with the presentinvention is able to detect moisture on the windshield of a vehicle inorder to automatically control the vehicle's windshield wiper, defrosterand/or defogging systems. The system for automatically sensing moistureon a vehicle windshield eliminates many of the performance deficienciesof known automatic moisture sensing systems at a commercially viablecost. As used herein, the term “moisture” is used to designate varioustypes of moisture and precipitation which can be found on the windshieldof a vehicle during various climatic conditions, such as rainfall,snowfall, ice, and fog, as well as other substances that are commonlydeposited on a vehicle windshield such as bugs, dust and the like. Thesystem is able to provide superior performance to other known systemsduring rather common climatic conditions, such as ice, fog and varyinglevels of rain and snowfall, and the like.

As will be discussed in more detail below, a portion of the windshieldis imaged onto an image array sensor. An optical system which forms aportion of the present invention causes raindrops and other sources ofmoisture on the windshield to be sharply focused while distant objectsbeyond the windshield are severely blurred in the image. The processingsystem analyzes the image for sharp discontinuities caused by the edgesof the water rain droplets or other moisture and by random focusing ofthe distant objects by the droplets. These discontinuities representhigh spatial frequency components. The magnitude of high spatialfrequency components is a measure of the amount of the rain or othermoisture on the vehicle which can be used to automatically control thevehicle windshield wipers. In an alternate embodiment of the invention,the system is adapted to sense fog on the interior and exterior of thewindshield in order to prevent spurious operation of the automaticmoisture sensing system. As such, the present invention eliminates manyof the various performance limitations of known automatic moisturesensing systems.

In yet another alternate embodiment of the present invention, the systemis able to detect operation of the windshield wipers in order to preventspurious operation and damage to the windshield wiper system duringconditions when the windshield wipers are stuck or frozen to thewindshield. As discussed above, the system analyzes the image of aportion of the windshield for sharp discontinuities which will haverelatively high spatial frequency components. The magnitude of thesehigh spatial frequency components is used to represent the measure ofmoisture or other substances on the windshield. Thus, dust, bugs andother substances will initially be treated as moisture. However, asdiscussed above, the system has the ability to automatically sense anoperation of the windshield wiper blades. Thus, if the substance, whichmay be ice, dirt, cracks or other substances not removable by thewindshield wipers remains on the windshield after one or more wipes, thesystem in accordance with the present invention may be configured toignore such substances in order to prevent further spurious operation ofthe vehicle windshield wiper system.

Referring to FIG. 1, the automatic moisture sensing system in accordancewith the present invention is generally identified with the referencenumeral 20. The automatic moisture sensing system may be mountedstationary in the mounting bracket 22 of an automobile rearview mirror24 or alternatively mounted in the rear portion of the rearview mirrorhousing 24. The automatic moisture sensing system 20 includes an imagesensor mounted, for example, 2–3 inches behind the vehicle windshield 26with the optical axis substantially parallel to ground or slightlyangled to the ground. The angle of the windshield 26 in a modernpassenger car is about 27°. Such a configuration may cause the raindropsand other moisture to be at a different distance from the image sensordepending on where the moisture is with respect to the field of view ofthe image sensor. To help compensate for this problem, the image sensormay be angled approximately 10° toward the windshield 26 such that thetop of the sensor 20 is moved closer to the windshield 26.

There are four main components to the automatic moisture sensing system20: an imaging optical system; a light emitting diode; an image sensor;and a processor. The imaging optical system is shown best in FIG. 2while the image sensor is illustrated in FIG. 2 and in FIG. 6. A flowdiagram for the microcontroller is illustrated in FIG. 5.

Imaging Optical System

The imaging optical system, generally identified with the referencenumeral 30 (FIG. 2), is used to image a predetermined portion of thewindshield 26 onto an image sensor 32 such that objects at theapproximate distance of the windshield 26 are sharply in focus at theimage plane while objects at a longer distance are out of focus andblurred. The area of the windshield 26 that is imaged must be largeenough that the probability of receiving raindrops during relativelylight rain conditions is significant. Moreover, the imaged area of thewindshield must also be in the area of the windshield that is wiped bythe windshield wipers.

The imaging optical system may include a single biconvex lens 33 used asan imaging lens. The lens 33 may have a diameter of 6 mm, a front andrear radius of curvature of 7 mm for each surface, and a centerthickness of 2.5 mm. The front surface of lens 33 may be positioned 62mm from the outer surface of the windshield 26. The imaging lens 33 maybe carried by a mechanical lens mount 34 which forms a stop 36 of about5 mm diameter directly in front of the lens 33. The image sensor may belocated about 8.55 mm from the rear surface of the lens 33 and asmentioned above, slightly angled by about 10°.

More elaborate optical systems, for example, with multiple elements,aspherical elements, or defractive objects could all be used, especiallyif shorter distance from the windshield is a desired feature. However,since the collected images are not for photographic purposes, suchoptical quality is not necessary in an application for automaticmoisture detection. A single lens may also be used molded in acrylic orother clear plastics at a relatively low cost. Various companiesincluding Polaroid and Kodak specialize in high performance moldedplastic optics.

FIG. 4 illustrates a computer simulation of the performance of theimaging system illustrated in FIG. 2. In particular, FIG. 4 a is a spotdiagram of the imaging of approximately parallel light rays from arelatively distant object on the optical axis onto an image plane. FIG.4 b is a spot diagram of the imaging of a point on the optical axis atthe distance of the outer surface of the windshield. Upon comparison ofthe spot diagrams of FIGS. 4 a and 4 b, it is evident that the opticalsystem is able to blur light coming from distant objects while focusinglight from objects at the windshield distance.

Occasionally, when driving up a hill, the vehicle could be positioned insuch a way that the sun is directly imaged by the device. The radiativeloading caused by this alignment may damage the image sensor 32 overtime. In order to alleviate such a problem, an electrochromic filter maybe used to temporarily eliminate most of the sunlight from the imageplane. Other optical electronic or optical mechanical devices could alsobe used.

Image Sensor

The image sensor 32 may be a CMOS active pixel sensor. CMOS active pixelsensors are a recent breakthrough in imaging technology that allow lowcost, high sensitivity imaging on a chip manufactured in a CMOS process.Such CMOS active pixel sensors have several advantages over othersensors including low power consumption, popular CMOS productiontechniques, low cost and ability to integrate additional circuitry onthe same chip, variable read out windows and a variable lightintegration time. Such CMOS active pixel sensors are commerciallyavailable from Photobit LLC, La Crescenta, Calif. While CMOS activepixel sensors have substantial advantages, other image sensors are alsosuitable and are considered to be within the scope of the presentinvention. The size and number of pixels are determined to image an areaof the windshield sufficiently large and in enough detail to adequatelydetect light rain while remaining cost effective. For example, a 64×64active pixel, 40 μm pixel size array will image approximately a 25 mm×40mm, on a standard passenger car windshield.

Processing and Control

A block diagram of the automatic moisture sensing circuitry is shown inFIG. 6. As mentioned above, a predetermined portion of the windshield 26is imaged onto an image array sensor 32. The analog voltage of each ofthe pixels within the sensor 32 is converted to digitized gray scalevalue by way of an analog to digital converter. The analog to digitalconverter 35 is operated under the control of a timing and controlcircuit 37 which, in turn, is controlled by a microcontroller 38. Thetiming and control circuit 37 is described in detail in U.S. Pat. No.5,990,469, entitled “CONTROL CIRCUIT FOR IMAGE ARRAY SENSORS,” by Jon H.Bechtel et al., assigned to the same assignee as the assignee of thepresent invention, hereby incorporated by reference. A suitablemicrocontroller 38 is a Motorola type 68HC08XL36. However, it iscommonly known that such microcontrollers do not contain sufficientrandom access memory (RAM) to store an entire image from a 50×50 pixelimage sensor. In such a situation, a windowing feature of the CMOSimaging sensors may be used to alternatively image and process differentregions of small enough size for the onboard RAM of the microcontroller38.

As discussed above, the system analyzes the digitized gray scale valuesfor sharp edges which are representative of raindrops or other moistureby analyzing the spatial high frequency components. The magnitude of thespatial high frequency components is used to control a windshield wipermotor control 40 such that the frequency of wiping of the windshieldwiper blades (i.e., time interval between wipes) is controlled as afunction of the amount of moisture on the windshield. As will bediscussed in more detail below, the system is also able to detect thefog on the interior and exterior of the windshield. Thus, themicrocontroller 38 may also be used to automatically control the vehicledefroster or defogging system 42. In order to provide selectivity of asystem, a driver on/off sensitivity control circuit 44 may be provided.This control circuit 44 may be used in special circumstances, forexample, when the vehicle is in an automatic car wash to preventspurious operation of the system.

Once an image is acquired by the image array sensor 32, the luminance oneach pixel, represented by an analog voltage, is converted to a digitalgray scale value by the analog digital converter 35. These values arewritten to memory, which may be on board the microcontroller 38 andprocessed by the microcontroller 38 or alternatively a digital signalprocessor.

Rain is detected by quantifying the discontinuity resulting from sharpedges of the raindrops on the windshield. These sharp edges are causedby the focused image of the rain or other moisture droplet along withthe random optical imaging of far field objects by the droplets or othermoisture. As discussed in “Digital Image Processing” by R. C. Gonolezand R. E. Woods, Addison-Wesely 1992, hereby incorporated by reference,the images may be analyzed in terms of their spatial frequencycomposition. Spatial frequency composition analysis is analogous toFourier analysis, commonly used in both digital and analog signalprocessing. The process of taking a Fourier transform of a signal anddetermining its frequency composition can readily be applied totwo-dimensional signals. When the two-dimensional signal is an image, itis common to use the term “spatial frequency.” The spatial frequencycomposition of an image can be evaluated using a two-dimensional Fouriertransform of the image. The transform is given by equation (1) asfollows:

$\begin{matrix}{{F\left( {\omega_{y},\omega_{y}} \right)} = {\int\limits_{- \infty}^{\infty}{\int\limits_{- \infty}^{\infty}{{f\left( {x,y} \right)}{\mathbb{e}}^{{- {j\omega}_{x}}x}{\mathbb{e}}^{{- {j\omega}_{y}}y}{\mathbb{d}x}{\mathbb{d}y}}}}} & (1)\end{matrix}$where: f (x,y) is the value of the pixel in the original image locatedat pixel x,y; F (ω_(x), ω_(y)) is the value of the Fourier transform ofthe image at pixel location (ω_(x), ω_(y)); and j is the complex number√{square root over (−1)}.

Equation (1) describes the Fourier transform for continuous infinitetwo-dimensional signals. This function can be readily adapted todiscrete, finite two-dimensional signals resulting from digital images.Applying spatial frequency analysis techniques, the rough edges or“roughness” of an image can be relatively accurately quantified. Forexample, a Fourier transform can be performed on a very blurry image. Insuch an analysis, the value of F (ω_(x), ω_(y)) for low magnitudes ω ofspatial frequencies ω_(x), ω_(y) will be high while the value of F(ω_(x), ω_(y)) at high magnitudes of ω_(x), ω_(y) will be low. The valueof F (ω_(x), ω_(y)) where ω_(x), ω_(y) are both 0 is always the averagegray scale value of the image.

Alternatively, a Fourier analysis of a sharply focused image with manyedges will result in the values of F (ω_(x), ω_(y)) for large magnitudesof ω_(x), ω_(y) being high. A digital filter can be used to selectparticular spatial frequency regions. A relatively simple implementationof such a filter for image processing uses a 3×3 matrix supplied to a3×3 pixel neighborhood as illustrated below:

A B C D E F G H IA new image g(x,y) can be formed which is the resultant image ofapplying the filter to the current image. The image may be processed ina looping fashion for every pixel with a value of f(x,y) at a locationdefined by the variables x and y. In the above matrix, the location ofthe coefficient E corresponds to the current pixel at x and y. The pixelat location x and y in the new image has a value given by equation (2)below:g(x,y)=A·f(x−1,y−1)+B·f(x,y−1)+C·f(x+1,y−1)+D·f(x−1,y)+E·f(x,y)+F·f(x+1,y)+G·f(x−1,y+1)+H·f(x,y+1)+I·f(x+1,y+1)  (2)

A special filter commonly used is a Laplacian filter. The Laplacian isthe second-order derivative of a two-dimensional function f(x,y) givenby equation 3:

$\begin{matrix}{{\nabla^{2}f} = {\frac{\partial^{2}f}{\partial y^{2}} + \frac{\partial^{2}f}{\partial x^{2}}}} & (3)\end{matrix}$This Laplacian function can be implemented in discrete space using the3×3 matrix described above with the coefficients as follows: E=4; B,D,F& H=−1 and the rest of the coefficients zero. Other coefficientcombinations can also be used to compute variations of the discreteLaplacian as long as the coefficient E is positive and the rest arenegative and that the sum of all coefficients is zero. The spatialfrequency response of any 3×3 filter is determined by equation (4)below:

$\begin{matrix}{{H\left( {\omega_{x},\omega_{y}} \right)} = {\sum\limits_{m = {- 1}}^{1}\;{\sum\limits_{n = {- 1}}^{1}\;{{{h\left( {m,n} \right)} \cdot {\mathbb{e}}^{{- {jw}_{x}}m}}{\mathbb{e}}^{{- {jw}},n}}}}} & (4)\end{matrix}$where:

H(ω_(x), ω_(y)) is the frequency response of the filter for frequenciesω_(x) and ω_(y); the function h(m, n) describes the coefficients of thematrix above; the coefficient E is the value of h(0,0), A is the valueof h(−1,−1), etc; and j is the complex number √{square root over (−1)}.

By analyzing the frequency response of the discrete 3×3 Laplacian filterusing equation (4), it is evident that the discrete 3×3 Laplacian is ahigh pass filter. By modifying the coefficients, the particular responseof the filter can be adjusted. Additionally, a 5×5 or larger filter canbe used for even finer control over the response.

Raindrops and other moisture can be detected by using a 3×3 Laplacianfilter described above. Every pixel is examined individually in a loopfashion and a variable used to store the total amount of “moisture”detected. The Laplacian for each pixel is computed using the formula forg(x,y) described above with the Laplacian coefficients.

A flow diagram in accordance with the present invention is illustratedin FIG. 5. Initially in step 46, an image of the windshield is acquired.As mentioned above, the optical system in accordance with the presentinvention images the scene in such a way that distant objects are out offocus and objects at the windshield distance are in focus. Thus, ifthere is no moisture or other matter on the windshield, only a blurryimage of distant objects will be captured. A blurry image will have arelatively low high frequency spatial component. Thus, the value of theLaplacian in such a situation will be relatively low. If there is rainor other moisture on the windshield, the drops will be in focus and theimage will contain relatively large high frequency components. Despitethe blurring of distant objects by the optical system 30, oncomingheadlamps from other vehicles which are relatively bright may contributea significant high frequency spatial component. In order to filter thiscomponent out, pixels which have a gray scale value above the saturationlevel of the analog to digital converter (i.e., pixels with a gray scalevalue at or near 255) may be skipped.

In step 48, the Laplacian of each pixel is computed and the results arestored. More particularly, if the magnitude of a Laplacian is above agiven threshold, indicating a large enough high frequency spatialcomponent to indicate rain or other moisture, this value is summed withthe values of the other pixels to indicate the total value of rain orother moisture which is compared against a threshold, which may be auser set threshold as indicated in step 50. If the sum of the Laplacianof each of the pixels is greater than the threshold as determined instep 52, windshield wipers are actuated in step 54. Otherwise, thesystem loops back to step 46 and acquires a new image of the windshield.

The threshold indicated in step 52 may be a fixed threshold or avariable threshold. In applications where variable threshold is used,the threshold may be a user set threshold implemented by a control knobor slide with a voltage output. This voltage output may then be sampledand converted to a digital value which is appropriately scaled forcomparison with the sum of the pixels which indicates the total amountof rain.

In order to prevent spurious operation of the system, the operation ofthe wiper blade is sensed in step 56. In particular, a small sub windowof the image array sensor may be selected to allow more cycles to beprocessed per second in order to detect a relatively rapidly movingwindshield wiper. An image of the wiper is acquired in step 56. Eachimage is processed using two one-dimensional high pass filters; one forthe vertical direction and one for the horizontal direction as indicatedin step 58. Since the windshield wiper will appear as a vertical line inan image, there should be a significantly higher high frequencycomponent in a horizontal direction than in a vertical direction. Thus,a vertical high pass filter may be implemented using a 3×3 matrixdescribed above with the coefficient E set to 2, coefficients B and Hset to −1 and the rest set to zero. A horizontal filter is implementedwith the coefficient E set to 2, the coefficients D and F set to −1 andthe rest set to zero. The sum of each component is tallied in the samefashion as used to compute the Laplacian discussed above. In step 60,the ratio of the horizontal component to the vertical component iscomputed. If the horizontal component is much greater than the verticalcomponent, a vertical line is assumed to be present in the imageindicating the presence of the windshield wipers.

If the wipers of the automobile are designed in such a way that thewipers are never approximately vertical when it crosses the rain sensingarea, the filters described above can be modified to accommodate such aconfiguration. For example, various other edge detection methods wellknown in the art of image processing can also be used. Additionally, ifthe wiper speed for the vehicle windshield wipers is so fast that itblurs slightly in the image for the necessary exposure time, thehorizontal filter can be modified to subtract the pixels two positionsto the left and right of the current pixel instead of the pixelsimmediately next to the current pixel.

After the wiper has cleared the image sensing area, as indicated in step62, another image of the windshield is acquired in step 64 for which theLaplacian is computed. This computation is used as a zero pointmeasurement that may be subtracted from all subsequent measurementsuntil the next wipe. In this way, long-term high frequency spatialcomponents in the image of a dirty windshield, cracks, scratches andfrozen ice will not contribute to the detected amount of rain.

If the windshield wiper is not detected within a given time frame, thesystem assumes that a malfunction has occurred, which can be caused as aresult of the windshield wiper being frozen to the windshield. Duringsuch a condition, the operation of the moisture sensor in accordancewith the present invention can be suspended for a period of time toallow the ice to thaw. If outside temperature information is available,freezing climate conditions can be taken into account to decide if thewipers are failing because of a mechanical malfunction or due to ice.

The system is also able to adapt to varying light levels. In particular,during selected cycles, the average gray scale value of the image may becomputed. If this value is high, indicating an over exposure to light,the image sensor integration time in the following cycle may be reducedto lower the average brightness. Similarly, if the light level is low,the integration time may be increased. In relatively dark conditions,some image sensors may not be capable of collecting enough light in areasonable time to adequately image moisture such as raindrops. In sucha situation, an additional illuminator may be provided to brieflyilluminate the area of interest from behind while the image is beingtaken. If the windshield of the vehicle is not highly absorbent toinfrared radiation, a near infrared illuminator can be used as long asthe wavelengths are within the detectable region of the image sensor. Aninfrared illuminator has the benefit of not being visible to the humaneye and thus not distracting to the driver.

Fog Detector

In order to eliminate many of the operational deficiencies of knownmoisture sensing systems, the system in accordance with an alternateembodiment of the invention includes a system for detecting fog on theinterior and the exterior of the windshield. As illustrated in FIGS. 2and 3, a light emitting diode (LED) may be used for detecting fog bothon the interior and the exterior of a vehicle windshield 26. Twodifferent embodiments of the fog detection systems are disclosed both ofwhich rely on the difference in the way the light from the LED 66 isreflected from the windshield 26 in the presence of fog. Fog detectionmay be done in alternate processing cycles with the rain detection. Inparticular, many cycles may be used for rain and moisture detectionbetween fog detection cycles due to the slow onset of windshieldfogging. At the beginning of the fog detection cycles, an image may beacquired using a window containing the expected location of the spots.

In the first embodiment, a light source that is either highly collimatedor focused to a point at the windshield distance can be used. The lightsource can be either infrared emitting or visible emitting dependingupon the absorption characteristics of the windshield. Infrared sourcesare preferable since they are not visible to the human eye and thereforedo not pose a distraction. An infrared LED 66 may be used along with alens 68 of a focal length equal to the windshield distance as generallyshown in FIG. 2. The LED 66 may be positioned a few millimeters abovethe main optical assembly and angled so that the projected light asshown in FIG. 3 is aimed at the position of the main optical axis on thewindshield 26. The LED 66 is initially turned off and an image is taken.Immediately following, the LED is turned on and a second image is taken.The difference between these images is used for spot detection. If nofog is present, the light will reflect from the windshield 26 at theSnell angle which will carry it well outside the field of view of theimage sensor 32. If fog is present, the light will be reflected byLambertian reflectance causing the fog to be imaged as a small spot 70(FIG. 3). Due to the angles of the windshield 26 and the light source66, outside fog will produce a spot 70 lower than a spot 72 produced byfog in the interior surface of the windshield 26. The location of thesespots 70, 72 may be used to represent the presence of interior and/orexterior fog on the windshield 26. If fog is present on the interior ofthe windshield, exterior fog detection is not possible. However, thislimitation is of no consequence because vision would be impaired anyway.The following truth table indicates the conclusions drawn from thepresence of each spot:

Exterior (Lower) Interior (Upper) Reflected Spot Reflected Spot ResultPresent — Exterior Fog — Present Interior Fog Not Present Not Present NoFog

In an alternate embodiment an infrared or, if necessary, a visible LEDis used. The LED must be either relatively small or be used inconjunction with a pin hole and aimed such that the light from the LEDreflects off the windshield 26 and onto an image sensor at the Snellangle. In such a configuration, two reflections will occur: one off theinside of the windshield and one from the outer surface of thewindshield by specular reflection. In this embodiment if fog is notpresent, the spots are reflected at the Snell angle and are visible bythe image sensor. If there is fog on the exterior surface of thewindshield 26, the spot from the interior reflection will be present,but the spot from the exterior refection will be a blur. The differenceimage is analyzed for interior fog. Initially a 3×3 Laplacian filter isused to create a new image which contains only the high frequencycomponents of the original image. In this way, the blurred reflectionfrom fog is illuminated. The spots are detected by taking the maximumvalue of a group of pixels in the expected region of each spot. A groupslightly larger than the expected size of the spot may be used in orderto correct for slight misalignments and for the fact that the Laplacianwill only preserve the edges of the spots where the high frequencycomponents are present. Also, slight fog may cause the spot to grow, butmay not require action. For each spot, if the value is reasonablygreater than zero, it is determined to be present. The following truthtable indicates the detection of fog for the alternate embodiment:

Exterior (Upper) Interior (Lower) Reflected Spot Reflected Spot ResultPresent Present No Fog Not Present Present Exterior Fog Not Present NotPresent Interior Fog

If the exterior fog is detected, the windshield wipers may be actuatedto help remove the fog or, if desired, a warning light can be used toindicate this condition to the driver. Interior fog detection can beused to automatically actuate the vehicle defrost or defogger systempreventing the driver from waiting until the significant fog hasdeveloped. More involved processing methods may include determining thesize of the spot by computing the distance between the horizontal edgesof the spot and thus having a measurement of the quantity of fog on thewindshield which can be compared to a threshold.

Obviously, many modifications and variations of the present inventionare possible in light of the above teachings. Thus, it is to beunderstood that, within the scope of the appended claims, the inventionmay be practiced otherwise than as specifically described above.

1. A fog detection system for detecting fog on a vehicle windshield,said system comprising: an image sensor; an optical system operative toimage at least a portion of the surface of the windshield onto the imagesensor; an illumination device for illuminating an area within the atleast one portion of the windshield surface, said optical system andillumination device configured to image reflections caused by fog on theinside and outside of the windshield in a different positions on saidimage sensor; and a processing system in communication with the imagesensor and operative to analyze images from the image sensor todetermine whether there is fog on the inside or outside of thewindshield based upon the spatial position of the imaged reflections onsaid image sensor.
 2. The fog detection system of claim 1, wherein saidprocessing system activates the windshield defroster when fog isdetected on the inside surface of the windshield.
 3. The fog detectionsystem of claim 1, wherein said processing system activates thewindshield wipers when fog is detected on the outside surface of thewindshield.
 4. A fog detection system for detecting fog on a vehiclewindshield, said system comprising: an image sensor; an optical systemoperative to image at least a portion of the surface of the windshieldonto the image sensor; an illumination device for selectivelyilluminating an area within the at least one portion of the windshieldsurface; and a processing system in communication with the image sensorand said illumination device and operative to selectively activate anddeactivate said illumination device and to analyze and compare imagesfrom the image sensor when said illumination device is activated anddeactivated to detect fog on the windshield.
 5. A windshield wipercontrol system for a vehicle comprising: an image sensor; an opticalsystem operative to image at least a portion of the surface of thewindshield onto the image sensor, said optical system is furtheroperative to blur distant objects in the imaged scene while sharplyfocusing objects on the windshield; and a processing system incommunication with the image sensor and operative to analyze images fromthe image sensor to detect moisture on the windshield and to activatethe windshield wipers when moisture is detected.
 6. The system of claim5 and further including a rearview mirror assembly adapted for attachingto a vehicle, wherein said image sensor is supported by said rearviewmirror assembly.
 7. The system of claim 5 and further including arearview mirror assembly adapted for attaching to a vehicle, wherein atleast a portion of said processing system is supported by said rearviewmirror assembly.
 8. The system of claim 5 and further including anilluminator for providing supplemental illumination across substantiallyall of the imaged portion of the windshield surface.
 9. The system ofclaim 5, wherein said processing system further operative to detectoperation of a windshield wiper and to obtain a baseline image of thewindshield immediately after it has been wiped by the windshield wiper,said processing system utilizing the baseline image to distinguishmoisture from bugs, cracks, dirt, and other debris on the windshield.10. The system of claim 5, wherein said image sensor is mounted in saidvehicle such that said imaging surface is inclined with respect tovertical.
 11. A windshield wiper control system for a vehiclecomprising: an image sensor; an optical system operative to image atleast a portion of the surface of the windshield onto the image sensor;an illuminator for providing supplemental illumination acrosssubstantially all of the imaged portion of the windshield surface; and aprocessing system in communication with the image sensor and theilluminator and operative to analyze images from the image sensor todetect moisture on the windshield and to activate the windshield wiperswhen moisture is detected.
 12. The system of claim 11, wherein saidilluminator emits infrared radiation.
 13. The system of claim 12,wherein said processing system detects an ambient light level andactivates said illuminator when the ambient light level falls below athreshold level.
 14. The system of claim 11 and further including arearview mirror assembly adapted for attaching to a vehicle, whereinsaid image sensor is supported by said rearview mirror assembly.
 15. Thesystem of claim 11 and further including a rearview mirror assemblyadapted for attaching to a vehicle, wherein at least a portion of saidprocessing system is supported by said rearview mirror assembly.
 16. Awindshield wiper control system for a vehicle comprising: an imagesensor; an optical system operative to image at least a portion of thesurface of the windshield onto the image sensor; and a processing systemin communication with the image sensor and operative to analyze imagesfrom the image sensor to detect moisture on the windshield and toactivate the windshield wipers when moisture is detected, saidprocessing system is further operative to detect operation of awindshield wiper and to obtain a baseline image of the windshieldimmediately after it has been wiped by the windshield wiper, saidprocessing system utilizing the baseline image to distinguish moisturefrom bugs, cracks, dirt, and other debris on the windshield.
 17. Thesystem of claim 16, wherein, when moisture is detected and a windshieldwiper is not detected, said processing system generates a control signalto suspend operation of the windshield wiper.
 18. The system of claim17, wherein said processing system generates the control signal tosuspend operation of the windshield wiper if said processing system doesnot detect the presence of the wiper within a predetermined time period.19. The system of claim 16, wherein said processing system detects thepresence of the windshield wiper by determining if a plurality of highfrequency components in the direction of wiper travel through the imagedportion of the surface is greater than a plurality of high frequencycomponents in a direction normal to the direction of wiper travelthrough the imaged portion of the surface.
 20. The system of claim 16,wherein said processing system compares images from said image sensorwith those analyzed immediately following detection of the windshieldwiper to determine whether the images are different.
 21. The system ofclaim 20, wherein said processing system generates a control signal todeactivate the windshield wiper if said processing system determinesthat the compared images are not different.
 22. The system of claim 16and further comprising a user interface for allowing a user to adjustthe sensitivity of said processing system to moisture when activatingthe windshield wipers.
 23. The system of claim 16 and further includinga rearview mirror assembly adapted for attaching to a vehicle, whereinsaid image sensor is supported by said rearview mirror assembly.
 24. Thesystem of claim 16 and further including a rearview mirror assemblyadapted for attaching to a vehicle, wherein at least a portion of saidprocessing system is supported by said rearview mirror assembly.